How Much Does Lookahead Matter for Disambiguation? Partial Arabic Diacritization Case Study

Abstract We suggest a model for partial diacritization of deep orthographies. focus on Arabic, where the optional indication selected vowels by means diacritics can resolve ambiguity and improve readability. Our diacritizer restores short only when they contribute to ease understandability during reading given running text. The idea is identify those uncertainties absent that require reader look ahead disambiguate. To achieve this, two independent neural networks are used predicting diacritics, one takes entire sentence as input another considers text has been read thus far. Partial then determined retaining precisely which disagree, preferring based consideration whole over more naïve reading-order diacritization. For evaluation, we prepared new dataset Arabic texts with both full vowelization. In addition facilitating readability, find our improves translation quality compared either their total absence or random selection. Lastly, study benefit knowing follows word in toward restoration reading, measure degree lookahead contributes resolving ambiguities encountered while reading. L’Herbelot had asserted, most ancient Korans, written Cufic character, no vowel points; these were first invented Jahia–ben Jamer, who died 127th year Hegira. “Toderini’s History Turkish Literature,” Analytical Review (1789)